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Abstract 

Astronomy: State of the Art is a massive, open, online class (MOOC) offered through Udemy by 
an instructional team at the University of Arizona. With nearly 24,000 enrolled as of early 2015, it 
is the largest astronomy MOOC available. The astronomical numbers enrolled do not translate 
into a similar level of engagement. The content consists of 14 hours of video lecture, nearly 1,000 
Powerpoint slides, 250 pages of background readings, and 20 podcast interviews with leading 
researchers. Perhaps in part because of the large amount of course content, the overall 
completion rate is low, about 3%. However, this number was four times higher for an early cohort 
of learners who were selected to have a prior interest in astronomy and who took the class in 
synchronous mode, with new content being added every week. Completion correlates with 
engagement as measured by posts to the online discussion board. For a subset of learners, social 
media like Facebook and Twitter provide an additional, important mode of engagement. For the 
asynchronous learners who have continuously enrolled for the past 15 months, those who 
complete the course do so quickly, with few persisting longer than two months. The availability of 
a free completion certificate had no impact on completion rates when it was added midway 
through the period of data analyzed in this paper. This experiment informs a new offering of an 
enhanced version of this MOOC via Coursera, along with a co-convened “flipped” introductory 
astronomy class at the University of Arizona, where the video lectures will be online and class 
time will be used exclusively for small group labs and hands-on activities. Despite their typically 
low completion rates, MOOCs have the potential to add significantly to public engagement with 
science, and they attract a worldwide audience. 
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Introduction 

Higher education in the United States is in a state of flux and stress. Two and four year higher 
education institutions are still seen as a gateway to a better future and financial security, but the 
cost of a college education has increased by a factor of two (inflation-adjusted) over the past 30 
years, and outstanding student loans of nearly a trillion dollars exceed credit card and automobile 
loan debt (Frye 2012). STEM (Science, Technology, Engineering, and Mathematics) education 
faces its own particular set of challenges. The science literacy of the average citizen is modest, 
leaving them poorly prepared to understand or judge technical societal issues like energy 
production, climate change, and genetic engineering (NSB 2012). Meanwhile, there is growing 
demand for graduates with technical skills and majors but the supply of trained graduates is 
inadequate. Completion rates for STEM majors are 40%, much lower than in non-technical 
subjects. The administration of President Obama has set a challenging goal of training 100,000 
new STEM teachers and producing a million more STEM graduates (NSTC 2013). 

The rapidly increasing capability of the Internet and the availability of online content are rapidly 
eroding the “sense of place” in a college education. Online learning is not new but it has started to 
become a major component of the educational landscape. The number of students taking one or 
more online class grew from 2 million to 7 million from 2002 to 2011 and over that period the 
percentage of total college enrollment online grew from 10% to 32% (Figure 1, Meeker and Wu 
2013). Reticence over the quality of online instruction is steadily diminishing; in a 2012 survey, 
77% of the academic leaders at 2,800 U.S. colleges and universities considered purely online 
education to be the same as or superior to face-to-face education (Babson 2013). Set against this 
rapid growth is the fact that research-based and learner-centered teaching methods have not been 
adopted widely in science classrooms (NRC 2012). The challenge of implementing effective 
pedagogy without face-to-face contact or personalized interaction is an unsolved problem. 


Massive Open Online Classes 

The instructional approach attracting the most media attention is MOOCs, or massive open 
online classes. The traditional MOOC is free, open to everyone, and does not result in university 
credit, so it is more appropriately considered outreach than formal education. Many universities 
are putting video lectures and entire courses online, and the number of iTunes U downloads has 
grown in four years to 60 million (Figure 1). Coursera is the largest company delivering online 
courses, and their course enrollment has grown at a meteoric rate, from 1 million in September, 
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2012 to 9 million in April, 2013. By lowering the bar for access to instructional materials and 
expert teachers, private companies like Coursera, edX, and Udacity are helping to democratize 
higher education. Companies like Udemy are making substantial contributions to professional 
development. For example, 75% of iTunes U downloads and 70% of Coursera users are located 
outside the United States (Masters 2011; Roller and Ng 2012). Coursera pioneered the delivery of 
online courses for transferable college credit. This opens up a new and powerful marketplace in 
higher education, where students can combine elements of a degree using local, face-to-face 
classes, and the best of the online offerings. MOOCs are often aimed at adult “recreational” or 
lifelong learners but a majority of the undergraduate population is now composed of students 
older than twenty-two, and these are often people with families and jobs. They will increasingly 
demand flexible delivery of the courses they are taking. 

Along with spectacular growth and obvious promise, MOOCs have been subject to skepticism and 
pushback. Administrators are fearful of the threat to the current, unsustainable cost model for 
higher education but they also hope it may be a way to spread their “brand” to new markets, while 
educators are excited by the potential of the Internet as a learning medium but are suspicious that 
the quality of learning will be poorer when it is online. Research on MOOCs is in its early stages so 
there is a need for analysis informed by data gathered on classes that have firm pedagogical 
foundations. Some research questions are difficult to address due to privacy issues in the use of 
learner demographics an online behavior, and due to limited data analytics provided by some 
MOOC hosting companies. The factors that make for a successful MOOC are starting to emerge 
(McAndrew and Scanlon 2013; Reilly 2013; Ebben and Murphy 2014), and first results from the 
large-scale experiments of Coursera and edX are being published (Breslow et al. 2013; Perna et al. 
2013). General metrics for performance and success of MOOCs across all subject are being 
developed for the first time (Klobas 2014). Completion rates are generally low, so understanding 
and increasing student engagement in the disembodied, online environment will be essential if 
MOOCs are to spread and flourish (Ramesh et al. 2014). A second, more profound issue is the 
distinction between engagement and learning. If the goal of a science class is to increase content 
knowledge and facilitate a high level conceptual understanding of science principles and process, 
which is much harder to facilitate and measure than mere engagement, particularly in an online 
environment (Reich 2015). 


Astronomy: State of the Art 

Astronomy: State of the Art is a MOOC specifically developed to study student participation in an 
online learning environment. The project aims to serve multiple audiences of learners. One is 
those self-motivated adults who take a free course out of interest in the subject and not to get a 
degree or for university credit; this is the traditional audience for a MOOC. Another is a set of 
college students who use the online environment for lectures and quizzes but whose classroom 
time is devoted to hands-on activities and group work; this is the “flipped classroom” model. A 
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third we plan to serve in the future is composed of students remote from the university offering 
the MOOC, who are paying to take a completely online class for transferrable university credit; 
this is the part of the landscape with the potential to disrupt the traditional economic model of 
higher education. Our group is developing tools to work with all of these audiences, as far as 
possible using methods that are interactive, centered on learners, and grounded in research 
methods. Astronomy is an attractive vehicle for science literacy and outreach because it is a 
dynamic subject with new discoveries announced almost weekly. The web site for the class is 
http://www.udemy.com/astronomy-state-of-the-art/. Enrollment is free. 

The first implementation of Astronomy: State of the Art was intended as an outreach vehicle for a 
broad public audience. All aspects of astronomy from planets to cosmology were included, but the 
emphasis was on exciting topics where the research progress has been rapid: large telescope 
construction, adaptive optics, Solar System exploration, exoplanets, black holes, dark energy and 
dark matter, and big bang cosmology. Udemy was selected as the provider of the free course in 
part because a course can be offered by a single instructor without an institutional commitment. 
The privately-held company is smaller than Coursera and edX but has also been growing very 
rapidly, reaching 5 million students and 22,000 courses in early 2015 (www.udemy.com). Most 
Udemy courses are on technical subjects and for professional development and the courses are 
asynchronous in the sense that a student can sign up for the course at any time and follow the 
material at their own pace. The company doesn’t offer the infrastructure for any testing beyond 
multiple choice questions. But the architecture and user tools for video content are robust and 
scalable and they were entirely adequate for this experimental course. The instructor for the 
course was a senior professor in the Department of Astronomy at the University of Arizona, who 
has taught many face-to-face astronomy courses at the introductory and advanced undergraduate 
levels (Cl). He was assisted by a half-time staff educational technologist (MW), and a half-time 
undergraduate teaching assistant (CA), who were responsible for creating and editing video 
lectures and weekly assignments, as well as communicating with the students on the course web 
site and using social media. 

Our goals in this project included creating the first MOOC that presented astronomy in terms of 
the cutting edge topics that define the research frontier. A MOOC can have a broader goal than an 
introductory college class, since it accesses a demographic that is typically more mature, with 
more people who take the class out of general interest than because of a university requirement. 
Adult learners do not need or expect to master the formal apparatus of physics and mathematics 
needed to comprehend the universe. But they can be given a conceptually high level treatment 
that lets them share in the excitement of research on black holes and exoplanets and cosmology. 
Another goal was to investigate the behavior of two cohorts: the pre-registered group who had a 
prior interest in astronomy and the continuously enrolling (and much larger) group encountering 
the class in the Udemy marketplace. A third motivation was to investigate correlations between 
various modes of engagement, such as accessing course materials and usage of social media. 
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Course Content 

This course has gone through two modes: one “active” and a subsequent “passive” mode. The 
course was initially designed as a seven-week class to be offered beginning in March, 2013. In this 
synchronous early phase, a series of lecture videos was released each week, along with self-guided 
assignments and additional material for students to explore on their own. The additional content 
consisted of reading material and podcast interviews with scientists at the University of Arizona 
engaged in research related to the weekly topic. The core content is a series of video lectures that 
total about 14 hours. Video was primarily filmed as first person “talking head” of the instructor 
shot against a green screen so that backgrounds could be added later, interspersed with relevant 
animations, images, and video clips. The footage was edited and rendered in Final Cut Pro 10. 
There were between five and seven videos for each week’s topic; each video lasted an average of 
ten minutes with the shortest being around five minutes and the longest less than 20 minutes. All 
videos were subsequently posted on YouTube so they would be available for people not enrolled in 
the Udemy course (https://www.youtube.com/user/AstronomySOTA). 

In addition to the videos, each of the seven major sections of content had a brief overview video, 
lasting 2 to 3 minutes. Additional material for each section included a downloadable pdf file of a 
Powerpoint slide presentation consisting of 80-130 slides, a reading 20-30 pages long taken from 
one of the instructor’s popular science books, and links to several 40-minutes podcast interviews 
with researchers and faculty at Steward Observatory and the Department of Astronomy. A quiz 
was created for the first section, and the rest of the assignments consisted of participation in the 
citizen science of the Zooniverse project (http://www.zooniverse.com). Participants could also 
communicate with the teaching team through question and answer and discussion threads on the 
course website, and through social media on Twitter (https://twitter.com/AstronomySOTA) and 
Facebook (https://www.facebook.com/AstronomySOTA). In addition, there have been live video 
discussions where students can have questions answered in real-time by the instructor. These 
sessions occurred weekly during the first seven weeks of the class, decreasing to once every few 
weeks subsequently. 

An astronomy MOOC does not have the same instructional goals or technological underpinnings 
as a conventional class where students are evaluated on both content and physical principles. As a 
result, the video lectures and ancillary material did not go into detail on some core topics in a 
college-level astronomy class: positional astronomy, radiation physics and the inverse square law, 
atomic structure, comparative planetology, the Hertzsprung-Russell diagram that describes 
stellar evolution, and the classification and structure of galaxies. Table 1 provides an outline of the 
course content. 
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Table 1 


Outline of the Content for Astronomy: State of the Art 


Section 

Title 

Topics and Description 

1 

Introduction 

Overview and Welcome 

2 

Exploring Our Universe 

Telescope design and limitations imposed by 
optics and the atmosphere, adaptive optics, space 
astronomy, detectors, large new telescopes, and 
the detection of gravity waves. 

3 

Understanding Our Solar System 

The habitability of Mars and moons in the outer 
Solar System. Comparative planetology and the 
systematic behavior of atmospheres and surfaces. 

4 

Discovering Extra Solar Planets 

Methods of exoplanets detection and 
characterization, and the definition of habitability. 
The search for Earth-like planets and methods for 
characterizing exoplanets. 

5 

Probing Distant Stars 

The role of stars in creating the biogenic elements. 
End states of stars, particularly supernovae, 
pulsars and neutron stars, black holes, and tests of 
gravity. 

6 

Inspecting Other Galaxies 

Evidence for dark matter in all galaxies, evidence 
for black holes in the centers of galaxies. Active 
galactic nuclei, structure formation, and types of 
galaxies. 

7 

Examining the Cosmos 

Hubble expansion, dark energy, evidence for the 
big bang, inflation and the early universe, and the 
multiverse scenario. 

8 

Uncovering Evidence for Life 

The origin and unity of terrestrial biology, 
extremophiles, the search for biology and 
biomarkers beyond Earth, the Drake equation, 
and the search for extraterrestrial intelligence. 


Participants and Participation 

Astronomy: State of the Art went “live” on March 26, 2013. To “prime the pump” for the new 
class, the course was announced on the Teach Astronomy web site, which was created by the 
instructor in 2011 to support General Education astronomy classes at the University of Arizona 
(http: //www.teachastronomv.coml . That site was at the time getting 300 unique visitors a day, 
the majority of which were not college students but were members of the general public with an 
interest in astronomy. The instructor also publicized the course in emails to over 600 astronomy 
instructors in the United States, and with flyers mailed to over 800 amateur astronomy clubs in 
the United States. These initiatives, plus use of social media to reach astronomy fans, probably 
reached as many as ten thousand people in the run up to the launching of the class, and by the 
time the class started over 700 had pre-registered on the Teach Astronomy web site. 
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As a result, 833 people were enrolled when the class started on March 25, 2013, and all of them 
can be presumed to have prior interest in astronomy since the class had not yet been advertised in 
the Udemy online “marketplace.” Enrollment stood at 1,383 when the class entered Udemy’s 
marketplace on April 2, 2013, and 1,991 were enrolled when the class ended its active mode (with 
new content being added weekly) on May 15, 2013. Given rates of enrollment before and after 
April 2, at least 70% and as many as 90% of the 1,991 enrolled on May 15 came via active 
solicitation. This early cohort is probably atypical when compared the more than 14,000 people 
who enrolled subsequently, since it preferentially includes people with a strong prior interest in 
astronomy. Comparison of the behavior of these two cohorts is the most important “internal 
experiment” reported in this paper. 

Soon after the launch of the course, it was approved to be part of the Faculty Project, a set of 
Udemy courses typical of a Liberal Arts college curriculum ( http://www.facultyproiect.org/) . 
During the second, passive or asynchronous phase of the course, students enrolled steadily and 
explored the online materials at their own pace. Enrollment has grown steadily since the course 
started with more than 1,000 new students adding the course every month (Figure 2). The course 
is in the top 100 for enrollment among over 13,000 courses in the marketplace of Udemy. The 
average rating for the class among 300 people who submitted an evaluation online is 4.95 out of 5 
stars. In terms of enrollment and student ratings, the course is in the top five of the nearly 200 
Liberal Arts courses offered by Udemy. The ratings suggest the course is meeting its goal of 
providing a high quality lifelong learning experience in astronomy for a geographically dispersed 
audience. 

The authors of this study have completed Human Subjects training at the University of Arizona 
and the project has institutional review board (IRB) approval to use course data for research 
purposes, following the proviso that the data does not have names or student identifiers attached. 
IRB training involves the investigators and authors of this paper studying material and passing a 
test on the ethical uses of human subjects data. The data in this study were provided by Udemy as 
a series of seven “snapshots,” with the most recent downloaded in June 2014. Random sampling 
of 600 student names as they enrolled suggest that about a third of them are from outside the 
United States (based on foreign alphabet characters in the names) and about a third are women 
(based on names with unambiguous associated gender). More detailed statistics come from 
Google Analytics, which can be used to track unique users, sessions, page views, and average 
duration of a session (Figure 3). 

Enrollment for the course has maintained a steady climb since the initial rollout and surge in 
numbers at the beginning. The rate of enrollment is approximately 900 students per month. The 
Udemy and Google statistics show the unusual nature of the first cohort who enrolled for the first, 
“active” mode of the class, from March 26 to May 15, 2013. Over this interval, on average each 
week saw the course host about 700 users, 1,800 sessions, 8,000 page views, and sessions that 
lasted about 18 minutes. Over the following three months the average numbers dropped to 350 
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users, 600 sessions, 2,000 page views, and sessions that lasted about 8 minutes. Over the 
subsequent eight months, the average weekly number of users grew to about 1,000, the number of 
sessions to about 1,300, and the number of page views to about 4,000. Given steady enrollment 
growth over this past year, the implication is that the students who followed the first synchronous 
cohort are much less actively engaged in the material. This impression is accentuated by the data 
on average time per session, which has fallen over the duration of the course from 18 to about 5 
minutes in the Spring of 2014. 

MOOC audiences are often geographically dispersed, and this course is no exception. From 
Google Analytics, about 45% of the users (and sessions) since March, 2013 have originated in the 
United States. The rest of the top ten in number of sessions are the United Kingdom (6.2%), India 
(5.0%), Canada (4.7%), Australia (4.3%), South Africa (2.4%), Mexico (2.3%), Brazil (1.5%), Spain 
(1.4%), and Columbia (1.2%). Intriguingly, Mexican users spend twice as much time per session as 
users in any other country. This effect has not been noted anywhere else in the limited MOOC 
literature and is possibly related to the generally poor bandwidth of access to the Internet in 
Mexico. English-speaking countries dominate the top ten (Figure 4a); among the next twenty 
countries in the list only one (Ireland) is English-speaking. A map of users shows representation 
from every continent (Figure 4b). A remarkable 174 countries are represented, including single 
users in Laos, Madagascar, Turkmenistan, Tonga, and Vanatu. MOOCs truly represent a global 
phenomenon. 

Although Udemy offers limited demographic data to instructors, a snapshot of the enrollment 
habits of the class was obtained by viewing the home pages of 500 participants. This showed that, 
at the time they first enrolled for Astronomy: State of the Art, 45% of the participants had only 
enrolled for this astronomy class, so it was their first exposure to a MOOC. But there was a long 
tail of people who were enrolled in many classes on Udemy. About 10% were signed up for 50 or 
more classes and a handful had signed up for several hundred classes. Since no financial 
commitment is required for the many free classes on Udemy, this illustrates the phenomenon 
where some people engage in “stamp collecting,” or signing up for many classes that they have no 
intention or time to complete. Again, the more committed nature of the early cohort stands out. 
Among the 400 students who had enrolled before the class formally started, 85% were only 
signed up for Astronomy: State of the Art and fewer than 10% had signed up for more than one 
other Udemy class. 

Completion and Engagement 

The classic MOOC sets the bar low for learners since there is no cost and no grades, with the 
corollary that the level of engagement can also be very low. Astronomy: State of the Art has a large 
amount of content, amounting to about 14 hours of video, 800 lecture slides, 500 pages of 
readings, and 15 hours of podcasts. The most sobering statistics come from completion metrics 
(Figure 5). Udemy measures completion as the fraction of the content viewed or accessed. For 
video lectures this is the percentage of the total video played, bearing in mind that the video being 
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streamed does not prove that it was actually watched. For lecture slides and readings, an 
algorithm converts the content into equivalent minutes of engagement. In this course, podcasts 
were accessed by links outside of Udemy so they don’t count as content in terms of completion. 
Just over half of the people who enroll never access the materials, consistent with statistics for 
many MOOCs. Of those accessing the class materials, 3.2% complete all of them, 4.3% complete 
half of them, and 7.3% complete a quarter of them. The rate of completion has remained fairly 
consistent over time with a completion rate of 25 students per month, or just a few percent of the 
number of students enrolling in the course each month. The astronomical number enrolled is 
largely misleading; participation statistics bring it down to Earth. 

The emerging research literature on MOOCs is beginning to identity the factors that affect 
completion and retention (Adamopoulos 2013). Factors causing high attrition include lack of 
time, lack of motivation, feelings of isolation and lack of interactivity online, poor preparation and 
skills, and perceived hidden costs to MOOCs (Khalil and Ebner 2014). Completion rates should 
only be evaluated in the light of students’ self-reported intentions, which also affects strategies for 
personalizing a MOOC (Reich 2014). One study of dropouts from a MOOC found that the time 
commitment required and a lack of challenging activities contributed to the poor completion rate 
(Gutl et al. 2014), but in common with other studies also determined that most students never 
had any intention of completing the MOOC when they enrolled. 

Course completion rates were noticeably different between the initial cohort who signed up 
during the “active” or synchronous portion of the course, and those who signed up during the 
“passive” phase. Of the initial cohort of 1,991 students who joined the class during the “active” 
mode, 7.5% completed the course. The cohort of students who joined the course during the 
“passive mode” has lower completion rate of 3.3%. In addition, the time to completion is 
noticeably different between cohorts (Figure 6). Students in the first “active” mode cohort took 
around seven weeks (median=5i days, mode=45-49 days) to complete the course content. This 
should come as no surprise, since the course content was released on a weekly basis for seven 
weeks. Students in this first cohort who joined slightly after the start date took a little less time to 
finish the course since the content was already posted when they arrived. By contrast, students in 
the second “passive” mode cohort have generally completed the course quickly (median=33 days, 
with the largest group of students in this second cohort completing the course content in less than 
five days (mode=o-4 days). What we find is that completion times for the first cohort are 
clustered more closely together, compared with students in the second cohort who were able to 
pace themselves, and completion times are more varied. Even though there were 10 days in 
between the day the final “active” content was posted, and the cutoff date for the first cohort, no 
one in the first cohort finished in less than 25 days, whereas with the self-paced students in the 
second cohort, nearly 50% of the students completed the course in that amount of time. 

Most people access the content sequentially, so a much higher fraction views the first and early 
modules of the class than views the later modules (Figure 7). The last two graphs can be used to 
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loosely categorize students of this MOOC into three groups: “phantoms,” who enroll but never 
access the materials, “dabblers,” who view a handful of video or completion a small fraction of the 
course, and “persisters,” who complete the course or view most of the course materials. In fact, 
research on one biology MOOC with Coursera showed that first week performance and social 
interaction is a strong predictor of final performance (Jiang et al. 2014). Even though the 
participation and completion rates are low, the high aggregate engagement can make a MOOC an 
effective vehicle for outreach (Figure 8). Early adopters were watching 90 minutes or more of 
content, and even the less engaged later cohort is on average watching 40-50 minutes of content. 
The aggregate after just over a year is nearly 400,000 minutes or 6,700 hours, equal to 10,000 
people watching a typical public lecture on astronomy, involving reaching audiences who might 
not otherwise be able to enjoy cutting edge science. MOOCs will undoubtedly play a role in 
combatting low levels of science literacy in the United States (Impey et al. 2011, 2012). 

While there is a natural explanation for the high degree of participation of the early students who 
enrolled in the class, there is no explanation for the slow decline in participation over the past six 
months, manifesting in lower completion rates and rates of watching lectures, and lower rates of 
posting question and reviewing the course on the Udemy site. In fact, starting in June, 2013, all 
those enrolled were able to get completion certificates if they viewed all of the materials. This fact 
was well-publicized by Udemy and by several posts from the instructor. Given that the great 
majority of students are far from completion, a certificate presumably did not act as sufficient 
incentive. However, the null result of a certificate as an incentive to completion is significant 
given that Coursera has made paid and validated certificates a core component of their business 
model. Another component of the course involved social media. Astronomy: State of the Art has a 
Twitter feed, a Facebook page, and a blog, and all three have been used to update students on 
recent developments in astronomy and keep them involved in the class. Facebook is the most 
useful tool; the rate of visitors to the course page mirrors the rate of visitors to Udemy (Figure 9). 
Due to the privacy restrictions on Facebook data, we could not correlate the use of social media 
with access to the class materials on the Udemy site. 


Next Steps 

Astronomy: State of the Art succeeded in its goal of drawing a large number of people to the 
subject of astronomy, and giving them an overview of a vibrant subject of scientific research. Over 
23,000 are currently enrolled, and the number increases by about a thousand each month. 
However, more than half of those enrolled never access the online materials and the completion 
rate has been low. This experiment with Udemy yielded valuable experience in video editing, 
conducting live sessions, boosting participation and interest using social media (particularly 
Facebook and Twitter), tracking metrics of engagement, and diagnosing and setting appropriate 
learner expectations. We have no prescription to address the issue of low completion rates (Reilly 
2012), and a proportion of phantoms and dabblers are inevitable with an online science class, 
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where people act initially out of curiosity and with no intention of a major commitment. 
Completion rate in this MOOC is correlated with a high prior interest in astronomy, which is 
unsurprising. But it is also correlated with engagement through both synchronous (live sessions) 
and asynchronous (Q&A threads, social media) channels, which suggests that the video lectures 
and online quizzes that form the pedagogy of most MOOCs are not sufficient. Learners will do 
better when they have multi-modal engagement with the material and the instructor. 

We plan a second experiment in 2015 with Coursera, with which the University of Arizona has 
recently become a partner. Coursera has superior infrastructure than Udemy for online evaluation 
and testing, and they have facilitated a growing base of instructional research (Roller et al. 2013). 
The Udemy MOOC will continue to be open to anyone who is interested. For Coursera, several 
additional audiences with professional or personal motivations for deeper engagement will be 
recruited. High school astronomy teachers will be recruited from a national pool of about 1,500 to 
enhance their content knowledge and for professional development credit (Berendsen 2005). 
Also, there are about 800 astronomy instructors at community colleges and two-year colleges, 
who teach a quarter of a million students per year (Fraknoi 2001). Most do not have a degree in 
astronomy; they will value the content rigor and research currency of this MOOC. The most 
dedicated members of these cohorts will be recruited as citizen scientists for projects in the 
course, and as learning assistants in subsequent offerings of the MOOC. There is very little 
control over the motivations of students who participate in a MOOC as a free-choice learning 
experience, just as museums have little control over the motivations of their customers (Bell et al. 
2009; Donovan and Woods 2001). But it is possible to control the degree to which participants’ 
expectations are met. One eventual goal of this project is to understand the motivations of MOOC 
participants, to shape their expectations for how the MOOC will serve their education needs. 

In the future, the classic MOOC, which primarily has an outreach function, will be co-convened 
with “The Physical Universe,” a Natural Sciences course taught at the University of Arizona that 
satisfies a General Education requirement for non-science majors. Using the same core material 
as Astronomy: State of the Art (with additional modules on the physics of radiation, atomic 
structure, energy, and gravity that are not necessary for the informal learners), the local course 
will employ a “flipped” model where the students access lectures and podcasts online but are in a 
face-to-face classroom two times a week for labs and hands-on activities, lecture tutorials, group 
discussions, and other research-validated tools for enhancing learning. A key goal in developing a 
“next generation online science class” is to migrate the most pedagogically effective methods of 
the face-to-face classroom to a completely online environment (Prather et al. 2011). A flipped or 
hybrid model gives students flexibility, uses the online medium for the aspects of instruction 
where interaction with an instructor isn’t required, and optimizes the scarce resource of time in a 
large classroom (Figure 10). The eventual goal is to recruit undergraduates from anywhere in the 
United States and award them transferrable credit for completing the class. The audience will be 
students who need a science class as a graduation requirement or who are piecing together a 
degree with a mixture of online classes and face-to-face classes from their local institution. 
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Figure 1. Freely available educational materials like video lectures on iTunes U and YouTube (not 
shown here but growing equally rapidly) are democratizing education. Within the private sector, 
Coursera has the largest, most rapidly growing enrollments (Meeker and Wu, Internet Trends, 
Dn Conference, May 2013). 
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Total Enrollment Over Time 



2013 2013 2013 2013 2014 2014 

Date 

Figure 2. Enrollment in Astronomy - State of the Art over the first 14 months of the online 
course. The initial surge of enrollment was from recruitment of likely audiences based on prior 
astronomy interest. The course entered the Udemy marketplace in late March, 2013. Enrollment 
bumps in May and June 2013 were due to cross-marketing within Udemy and the rate of 
enrollment has shown grown steadily over the course of the past year to be twice as high as it was 
near the start of the course. 
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Figure 3. Data from Google Analytics for the course home page, showing number of user sessions 
for the first year. The engagement rate dropped immediately following the synchronous phase of 
the class, and subsequently rose, but not nearly at the rate at which new students enrolled in the 
class. This, plus average sessions times of 10 minutes across all users, shows that most students 
do not actively engage with the content or that many students have modest engagement. 
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Figure 4. (a) The top ten countries where the largest number of students taking Astronomy - 
State of the Art live, from Google Analytics. Relative to their populations, the United States, the 
U.K., and Canada are about equally represented and three of the top ten countries are Spanish¬ 
speaking. (b) A world map of students in the course shows representation from every continent 
except Antarctica, 174 countries in all. Some students manage to take the class even though they 
live in places with poor Internet infrastructure. 
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Student Completion Ratios 


7,500 



Figure 5. Student completion statistics after just over a year of operation of the MOOC 
Astronomy - State of the Art. Nearly 7,000 out of the nearly 13,000 enrolled never access the 
materials so they are “phantom” students. A large number access less than 20% of the materials; 
enough to get a flavor of the subject. A small percentage complete more than half of the materials, 
and the last bin shows an increase from the dedicated cohort who persist to the end of the course. 
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Completion Time 



Figure 6. Number of days taken by students to complete the course content. Students in the first 
cohort (active phase) took around 7 weeks to complete the course, similar to the length of the 
active phase. Students in the second cohort (passive phase) finished much more quickly, with the 
largest group completing the material in less than five days. 



Figure 7. Number of students completing a particular lecture in the course compared with the 
number of students viewing or partially completing a lecture. Most of the lectures are short (8 to 
12 minutes) videos but Udemy lists slides and readings as lectures too, and the dips in completion 
at the end of each section of the class correspond to lower completion of the slides and readings 
than the videos. Persistence can be seen in the flattening off of participation and completion for 
people who progress through more than the first quarter of the course. 
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Average Content Consumed 

110 - 



Overall Content Consumption 



Figure 8. (a) Average content consumed per month of the Astronomy - State of the Art course. 
The early cohort consisting primary of amateur astronomers and educators had a higher level of 
engagement that all the cohorts that followed. The reason for the decline in the past six months is 
not clear, (b) Total content consumed, showing the high engagement of the early cohort, and then 
a continuing rise as the enrollment grows but with a lower typical level of engagement. 
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Total Page Likes 


Figure g. Total number of page “likes” per day. Rate of “likes” closely follows the rate of student 
completion. 


NEXT GEN ONLINE SCIENCE COURSES 



Figure 10. A schematic view of the combination of informal and formal learning that is possible 
when online content is used to serve both audiences. The MOOC is a purely online experience, is 
offered free, and is not for college credit. Meanwhile, undergraduate students needing a General 
Education science requirement at the University of Arizona and elsewhere view online lectures, 
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and use the face-to-face time for active engagement, with learning facilitated by peer assistants 
from within the class. 


© Impey, Wenger, and Austin 

Athabasca University^ 
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